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ABSTRACT 



The purpose of this study was to document the iterative 
development of a design-based science curriculum called Learning Science by 
Designing Artifacts (LSDA) . The study refers to the enactment of the Safer 
Cell Phones curriculum in a high school located in the Midwest. The 
curriculum was a 5- or 9-week unit in an 18-week science elective course. In 
the initial version of the LSDA curriculum, students were actively engaged in 
the creation and presentation of their ideas while learning about the field 
of design and science content. However, the researchers had concerns 
regarding how well the curriculum was supporting the students in learning 
science. During this development process, the development team underwent a 
significant shift in their understanding of the theoretical framework called 
design-based science (DBS) . Over time the researchers changed their ideas 
toward having students act as product designers using a design cycle 
instructional framework that incorporated science content. This paper 
documents how the changes in the curriculum design and the researchers' 
evolving understanding of DBS are informed and supported by data on student 
learning and student motivation in the classroom. The results of this work 
have implications for future reform-based DBS curricula. (Contains 15 
references.) (PVD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



r 



I PERMISSION TO REPRODUCE AND 

DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



yBEEN GRANTED B 



2x 



r 



DEPARTMENT OF EDUCATION ' 

pm iPATin ucatlonal Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION i 
U f Tl ,. ^ CENTER (ERIC) 

AJDms document has been reproduced as j 

/ received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



! TO THE EDUCATIONAL RESOURCES 

I**- INFORMATION CENTER (ERIC) 

CN [ 1 

00 
in 



1 * 0f y!? w or options stated in this i 

: document do not necessarily represent ! 

9^ ,cial OERl position or policy j 

.... ‘ J 



Q 

w 



Learning Science by 
Designing Artifacts 
(LSDA) - A Case Study 
of the Development of 




Curriculum 

by 

Rachel Mamlok 
Charles Dershimer 
David Fortus 
Joe Krajcik 
Ron Marx 



2 

BEST COPY AVAILABLE 






Learning Science by Designing Artifacts (LSDA) - A Case Study 
of the Development of a Design-Based Science Curriculum. 

Rachel Mamlok 

- Weizmann Institute of Science, Rehovot, Israel 

Charles Dershimer, David Fortus, Joe Krajcik, Ron Marx 

- University of Michigan, Ann Arbor Michigan 

Paper presented at the National Association for Research in Science Teaching, 
March 27, 2001 , St. Louis MO. 



INTRODUCTION 

Reform in science education is a complex process. Successful innovations 
require the integration of a variety of different educational activities while 
addressing the system undergoing reform. James Gallagher (2000) comments 
that current “policy makers and scholars are recommending multidimensional 
actions that address an array of entities” to produce results in reform (pg. 399). 

He notes that issues such as teaching all students, inclusion of socio-scientific 
content, involvement of computers, and curricula that place greater instructional 
emphasis on scientific processes and on the history of development of scientific 
theories are important components of science education reform (Gallagher, 

2000 ). 

However, research on current instructional reform efforts highlights the 
multifaceted difficulties that accompany science education reform. Successful 
reform cannot be based on an innovative curriculum alone. The reform effort 
must address several issues related to the organizational culture of schools, the 
capability of the educators involved in terms of their understandings and 
expertise, and the policy and management structure concerns of the school 
system being affected (Blumenfeld et al., 2000). Successful reform requires 
innovators to study and implement multiple issues simultaneously in order to 
understand how to create a capacity to sustain the reform and increase the 
chances of success (Blumenfeld et al., 2000). 

Over the course of the 1999-2000 school year, the Center for Highly 
Interactive Computing in Education (hi-ce) at the University of Michigan worked 
to develop reform-focused science curricula, hi-ce has a history of involvement in 
science education reform in both urban and suburban settings. The development 
of the science curricula was one component of a larger project called the Primary 
Sources Network (PSN), a federally funded Technology Innovation Challenge 
Grant. The PSN project is working to create both science and social studies 
curricula that meet several goals. One goal of the project is to create a 
framework for understanding how primary sources can be used to support 
learning in high school classrooms and to adapt, design and develop learning 
technologies to support the use of primary sources. These innovative science 
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curricula address several of the reform issues noted above by Gallagher. They 
new curricula involve topics that are of popular interest to most students, they 
have students learning through the use of computers, and they have students 
learning standards-based science content while studying and designing artifacts. 

The purpose of this study is to document the iterative development of this 
Design-Based Science curriculum (LSDA - Learning Science by Designing 
Artifacts). During the development process, the development team underwent a 
significant shift in their understanding of the theoretical framework called Design- 
Based Science (DBS). Initially, we created curricula that had students learning 
about design as a set of Science-Technology-Society (STS) content topics. Over 
time we changed our ideas toward having students act as product designers 
using a design cycle instructional framework that incorporated science content. 
This paper will document how the changes in the curriculum design and our 
evolving understanding of the theoretical framework called DBS are informed and 
supported by our data on student learning and student motivation in the 
classroom. The results of this work have implications for future reform based 
DBS curricula. 

OBJECTIVES 

In the initial version of the LSDA curricula, enacted during the first 
semester of the school year, students were actively engaged in the creation and 
presentation of their ideas while learning about the field of design and science 
content. However, we had concerns regarding how well the curriculum was 
supporting the students in learning science. Our initial data from interviews 
revealed that students enjoyed the curricula, and the attitude questionnaires 
indicated that they had positive attitudes about science while using the curricula. 
However, we felt that too much time was being spent on explicitly teaching about 
the design process, leaving less time to deal with science than we would have 
liked. We then engaged in a process of redesigning the curriculum and gathering 
data to address our research objective: 

Can we redesign the curriculum to improve the depth of science 

content without causing a decline in students' attitudes towards the 

curriculum and towards science? 

Our research involved comparing the different versions of our curriculum 
focusing on students’ attitudes and students' content knowledge. We proposed to 
make changes to the second semester's enactment of the curriculum that would 
remove the lessons about the discipline of design, and increase the depth of the 
science lessons. Classroom activities would still include the process elements of 
design: working with design specifications, creating and revising models and 
drawings, and receiving feedback on work through pinups and critiques. These 
curriculum modifications came about as our understanding of the innovative 
instructional process Design Based Science evolved and as we created materials 
that better matched what was required for it to be successfully enacted. 
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THEORETICAL FRAMEWORK 

The reform-focused science curricula developed by the hi-ce group at the 
University of Michigan are founded on a social constructivist perspective 
(Blumenfeld et al. , 1997; Singer et al., 2000). These curricula can be 
characterized by four prominent learning features: 

a) active construction, which states that students must be actively 
engaged in explaining, generalizing, hypothesizing, representing, etc., 
in order for deep understanding to develop; 

b) situated cognition, which states that learning occurs best when 
contextualized in situations meaningful to the students; 

c) learning occurs through repeated exposure to the practices of a 
community of practitioners, and 

d) that communities of practitioners are essentially communities of 
disciplinary discourse. 

In addition to incorporating these learning features, the Learning Science 
by Designing Artifacts (LSDA) curricula have three main characteristics: they 
incorporate the use of primary sources, they use the Design-Based Science 
(DBS) framework, and they make extensive use of existing, adapted and new 
computer technologies. 

Design-Based Science (DBS) is a particular form of Project-Based 
Science (PBS) in which students’ activities focus on the need to design artifacts 
that are related to real-world problems chosen to be interesting and challenging 
to the students. For example, the design goal of the curriculum with which this 
research deals is, Can I design a cell-phone that minimizes radiation, battery, 
and sound hazards without compromising customer appeal? In DBS, the design 
of the artifacts is not a culminating activity at the end of the curriculum, but rather 
it is the framework around which all the learning activities are organized. 

The cell-phone curriculum begins by giving the students design 
specifications that specify the requirements that their cell-phones will have to 
meet. Having the kids watch a video of an ABC 20/20 story on cell-phone safety 
sets the context of cell-phones as potential sources of health and environmental 
hazards. The kids then read a newspaper article on the same subject and a 
classroom discussion is held about the facts and the opinions expressed in the 
video and in the newspaper article. The students then engage in a series of units 
that are organized around the different stages of the design process. Students do 
Internet research on the different components of a safer cell-phone 
(environmentally friendly batteries, safe sound levels, and reports on 
electromagnetic radiation levels). Students examine a museum collection of 
phones to summarize the changes in the form and function that telephones have 
gone through since their invention. Students collect data from teenagers about 
their daily use of phones and summarized this in a report. There are laboratories 
on building a battery and studying battery chemistry, building a speaker and 
studying sound waves, and learning about electromagnetic radiation from a 
series of demonstrations using TV antennas and a broadcast signal. Throughout 
this process, the students are participating in the design studio process by 
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utilizing cycles of production and reflection. Students express their ideas through 
drawings and written essays and then get feedback on their ideas in pin-up and 
critique sessions. 

Other than the Science by Design series developed at TERC (TERC, 
2000) and the Learning Science by Designing Artifacts (LSDA) curricula created 
by the hi-ce group at the University of Michigan, we are unaware of other high 
school Design-Based Science curricula that have been developed in the United 
States. A middle school DBS curricula called Learning By Design (LBD) has 
been developed at Georgia Tech (Kolodner, et al., 1998) and a program that 
engages 4 th and 5 th grade students in creating science-oriented computer 
simulations has been developed at UCLA (Kafai and Ching, 1998). A number of 
other researchers write of using design in primary school classrooms (Roth, W- 
M., 1996; Penner, D.E., et al., 1997), but these’authors are more interested in the 
development of models by the students and the importance of these artifacts in 
the learning process; they are not explicitly attempting to use design as a vehicle 
for teaching science. 

As mentioned earlier, after the first enactment of the cellphone curriculum, 
we felt that too much time had been spent learning explicitly about the design 
process in comparison with the time spent learning about science. We decided 
to modify the curriculum, to try and make the design process a part of the 
pedagogical framework without having to deal with it explicitly, freeing time to go 
deeper into the science content, but still allowing the students to engage in a 
design process. 

The curriculum went through two major revisions in content, activities, and 
organization. Each revision was based on a better understanding on our part of 
how to incorporate science content into a design framework. The table in 
appendix A summarizes the main differences between the three versions. In the 
first version of the curriculum, students spent as much time learning explicitly 
about design as they spent learning about science. In the second version more 
time was given to science, while in the third version almost all the lessons about 
design were dropped. Instead, a cycle we call the DBS Learning Cycle (figure 1 
below), which is structured around the design process, became the framework 
for how classroom activities progressed. This allowed the students to engage in 
a design process without having to be explicitly taught about this process. 
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Figure 1: The design based science (DBS) learning cycle 



As the DBS Learning cycle shows, the students are actively engaged in 
constructing knowledge by asking and refining questions, developing personal 
ideas, designing and building models, conducting investigations, gathering, 
analyzing and interpreting information and data, drawing conclusions and 
reporting out their findings. Every cycle is situated and driven by a 
contextualizing activity. In design sessions,- classroom members continuously 
share their questions and design ideas with their group partners, while in pin-ups 
they share their solutions with their classmates, helping to generate a sense of 
belonging to a community of designers. In feedback sessions, while sharing and 
critiquing of each others ideas, the students are encouraged to engage in 
disciplinary discourse, not unlike what would be found existing ampng a 
community of professional designers. 

SITE: 

This study refers to the enactment of the Safer Cell Phones curriculum in 
the 1999-2000 academic year in a high school located in the Midwest. The 
school was a small public charter school situated within a museum dedicated to 
the role of innovative American technologies and the stories of the people 
responsible for these inventions. Students participate in a traditional curriculum 
consisting of mathematics, science, English, civics, art, and physical education. 
The Safer Cell Phone curriculum was a nine or five week unit in an 18-week 
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science “elective” course and all students in the 9 th grade were assigned to take 
the course. This “elective” course was taught during both the first and second 
semesters. 

POPULATION: 

75 students participated in the study with approximately 55% African 
American, 37% white, not of Hispanic origin, and the remaining students being 
Asian, Hispanic, Arab and multiracial. 

Two teachers participated in the study. One was an experienced math 
instructor who held a master's degree in education, the other was a first year 
teacher who had a Bachelor's degree in Science and had obtained his 
certification after his bachelor's degree. Both teachers' team enacted the 
curriculum in its first version; the less experienced teacher enacted the second 
and third version. Both teachers were part of an informal professional 
development program that addressed understanding Design Based Science, and 
both teachers were part of the curriculum development team and participated in 
the process of modifying and refining the curriculum. 

METHODOLOGY: . , 

Quantitative methods consisted of science content knowledge pre and 
post-tests, and pre and post attitude questionnaires. Qualitative methods 
included observations of the classroom, video documentation of specific lessons, 
and interviews with target students. The quantitative data was collected for each 
of the three versions of the curriculum. The qualitative data was collected for the 
first and third iterations of the curriculum, but are not reported in this paper. 

The attitude questionnaire was made up of two parts. The first was a 
semantic differential questionnaire that consisted of 15 semantic differential 
scales. It was construct validated by using a factor-analytic investigation (varimax 
rotated). Three factors were retained, covering 62% of the total variance. These 
three factors addressed affective, cognitive, and applicability attitudes. These 
items are reported in Appendix 2. Each item is a member of a subgroup for a 
difference construct, these constructs are reported in Appendix 3. The second 
part consisted of a questionnaire containing 47 items aimed at probing students’ 
attitudes towards science and science learning. The items were adapted from a 
work done by Tal and her colleagues (Tal et. al., 2000) based on the work of 
Midgley and her colleagues (Midgley et. al., 1998). The scale measuring 
students’ attitudes was a Likert type, 1-5 scale inventory (in which 1=not at all 
true while 5=very true). Example items are reported in Appendix 4. Each item is 
a member of a subgroup for a different motivation and attitude construct and 
these are reported in Appendix 5. 

The content knowledge test consisted of 15 multiple-choice questions and 
5 open-ended questions. The test probed for different levels of comprehension 
using low, medium, and high cognitive demand items that focused on the specific 
science content and the design process that was addressed in the curriculum 
(Krajcik, et al. 2000). The multiple-choice questions were in the low and medium 
cognitive demand category; the open-ended questions were in the medium and 
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high cognitive demand category. This test was administered by the teachers as 
both a pre and post-test in order to describe the knowledge that the students in 
the study held and gained. Example items are reported in Appendix 6. 

METHODOLOGY ISSUES 

We have two concerns with our methodology. These issues are being 
reported because we believe that these issues impact the results of the data 
analysis and the interpretation of these results: 

Population Differences 

Version one of the curriculum was enacted during semester one of the 
1999-2000 school year. The 50 students participating in the enactment of version 
one were entering 9 th graders who were new to high school and to each other. In 
addition, these 50 students were grouped together based on their scores on a 
mathematics achievement test taken before school started. Many of these 
students scored below the 9 th grade level, and were placed together to receive 
some math content remediation in a mathematics elective course. 

Version three was enacted in the latter part of the second semester of the 
1999-2000 school year. These students had already taken the required ninth 
grade mathematics and science course during the first semester of the school 
year. In addition, the instructor that taught the safer cell phone unit was the ninth 
grade science instructor. The concepts of electricity, circuits, and battery 
chemistry were covered by this instructor in the 9 th grade science course, 
including some of the same laboratory activities. These students were grouped 
together because many of them scored at or above grade level on the 
mathematics achievement test taken before entering 9 th grade. 

Science Content Test Differences 

Due to the changes to the curriculum, several items on the science 
content test needed to be modified or replaced. These questions either dealt with 
topics that were not addressed in the new version of the curriculum, or they were 
worded or structured poorly. Since the teachers used the post-tests as their unit 
exams, in order to assist the teacher in giving a course grade, a decision was 
made to revise the post-test of the third revision, thereby making the test a better 
measure of student learning. This meant that comparisons could not be made 
between one multiple-choice item and all of the open-ended questions for version 
1 and version 3 of the curriculum. 

DATA ANALYSIS METHODS 

Paired t tests (one tailed, alpha = .025) were run for the total pre and post 
test mean scores of the content tests and on the means scores for each item of 
the pre and post content tests. Effect size was calculated for the total mean pre 
and post-test differences. For the attitude questionnaires, means were calculated 
for each sub grouping of questions, and paired t tests were run for these means. 
When comparisons were made between version one and version three pretests, 
post-tests, or attitude subgroup means, unpaired t tests were performed. . 
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FINDINGS FOR THE VERISON 1 CURRICULUM ENACTMENT ( Semester 1) 
Attitude Surveys: 

For Part One of the attitude questionnaires, the semantic differential items, 
significant differences were only seen on the cognitive grouping of items (p< .05), 
with no differences on the affective and applicability of science groupings. For 
Part Two, the Likert scale questionnaire, there were significant improvements on 
three subgroups; Personal Mastery Goal Orientation towards learning in science 
(p<.01), reliance on Extrinsic goals in science and Science Attitudes (p<.05). 
There were no differences between the pre and post questionnaires on the other 
nine subgroups. The implications are that this group of students ended the unit 
with higher self-ratings for science being “easy, simple, and clear.” These 
students end the unit rating themselves higher on their personal interest towards 
learning science and their self-directed learning while in science class. 
Interestingly, they also rated themselves higher on their interest in getting good 
grades and pleasing the teacher. Their positive attitudes towards science 
increased from the start to the end of the unit. 

Science Content Tests: 

For the science tests the total mean scores were calculated and a 
comparison of means was made after removing the unmatched data' (n=47). The 
results identified a significant difference (p<.001) between the pre and post-tests 
total mean scores. The Semester 1 pre-test mean was a 4.10 with a standard 
deviation of 1 .83 and the post-test mean was 6.70 with a standard deviation of 
2.06. The calculated effect size for this difference was 1 .44 SDs. 

FINDINGS FOR THE VERISON 3 CURRICULUM ENACTMENT (Semester 2) 
Attitude Surveys: 

Students ended this unit with no significant differences on any of the 
groupings of the semantic differential, and on any of the 12 subgroups for the 
Likert scale questionnaire. While there were no significant improvements, there 
are also no significant decreases, indicating that the changes in the amount of 
science content included in the curriculum did not affect student attitudes toward 
their interest in and learning about science. 

Science Content Tests: 

For the science content tests, the total mean scores were calculated and 
unmatched data was removed (n=23). However, this time the total scores were 
calculated using 14 of the 15 items because one item was modified between the 
administration of the pre and post-test. In the second semester, the pretest 
mean score was 7.70 with a standard deviation of 2.87. The post-test scores 
mean was 9.57 with a standard deviation of 1 .73. The paired samples t test 
yielded a significance between pre test and post test means scores (p =.001). 
The calculated effect size for the semester 2 results was .651 SDs. 
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Analyzing the pre and post score mean difference by item revealed some 
results that may help explain the lower effect size. Significant differences 
(p<.001 ) were seen on five items. These items addressed the concepts of 
describing electron flow, identifying examples of electromagnetic waves, 
explaining the potential difference in a battery, identifying a correctly built circuit 
(a TIMMS item) and a process question identifying the different stages in the 
design process. Five items yielded no statistically significant difference; three 
items had a high pre-test score and two items had no difference between pre and 
post-test scores. The high pretest scores dealt with defining what a cellular 
phone is, defining the type of energy change in a battery, identifying what form of 
electromagnetic radiation causes sunburn, and describing how the radiation in a 
cellular phone is different from that of a microwave. Three items showed a 
decrease between the pre and posttest score; one item was on the testing of 
designed objects, and two items addressed how the model of the atom is used to 
explain electrical flow in a wet cell. The design question addressed a design topic 
that had been removed from version three of the curriculum and perhaps the 
content of this question was confusing to students at the end of the unit. The 
reasons for decreases on the two items addressing electrical flow in a wet well 
will be addressed in the teacher capabilities of the results and discussion section 
of this paper. 

RESULTS AND DISCUSSION 

The third version of the Safer Cell Phone curriculum was redesigned in 
order to provide more depth and detail to the science content while spending less 
time on the content associated with the discipline of design. The process 
elements of design that remained in the curriculum included the design goal, the 
cycles of revision and reflection, and the pin-ups and critiques. These design 
studio process elements were not taught as separate lessons, but integrated into 
the daily classroom activities of the curriculum. The science content was 
increased by adding a new battery laboratory, an electromagnetic wave 
laboratory, readings on both of these topics, questions about the readings, and 
short answer questions and quantitative problem sets concerning the science 
principles associated with the concepts of battery chemistry, and electromagnetic 
waves and energy. 

The results we obtained support our desired goal; we were able to 
increase the science content in the curriculum without decreasing student 
interest in science or the interest in the units. In addition, significant content score 
gains were achieved on the science content tests for both versions of the 
curriculum. However, we are left with an issue to address. 

• Why did we not see a larger increase in learning on the post-test 
scores that reflects our increase in the depth of the science content 
added to the curriculum? 

In the following discussion, we will explore these issues by examining three key 
differences between the curricular enactment's; Student prior knowledge 
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differences between groups, differences in student motivation when engaging in 
project-based work and differences in teacher capabilities, and differences in 
school policies and management. 

Differences due to prior knowledge 

Students were not randomly assigned to the different versions of the 
curriculum. The school administration decided to sort students into the first and 
second semester elective course based on the scores that students received on 
a math achievement test taken prior to entering the 9 th grade. Many of the 
students who participated in the enactment of the 3 rd version of the curriculum 
scored at or above their grade level on this test. We might then infer that these 
same students would also be high achievers if they were to have taken a science 
achievement test. While we cannot support this math and science achievement 
correlation empirically, we can examine and compare the science content pretest 
scores between groups. 

Pre-test score comparisons indicate entering knowledge differences 
between the version groups. Version 1 had a pre-test mean of 4.10, SD = 1.8 
and the version 3 students had a pre-test mean of 7.70, SD = 2.87. It is possible 
that we did not see larger gains for these students because their starting scores 
did not provide much room for gain on our science content test. Larger gains on 
the version 3 post test may have required a larger cost in terms of student 
instructional effort, teacher capabilities regarding the science content, and the 
time provided for the instructional tasks. The degree that these issues may have 
impacted the potential for post-test gains will be discussed in the next sections. 
Differences due to student motivation 

Many of the components of the DBS curriculum are similar to the 
components of project based science curricula. Students have design goals 
similar to driving questions, students investigate real world phenomena, students 
create artifacts, students collaborate, and students utilize technology (Krajcik et 
al., 1998). Blumenfeld, Soloway, Marx, Krajcik, Guzdial, and Palinscar (1991) 
make this comment about student motivation in project-based science; 

“It is insufficient to provide opportunities designed to promote 
knowledge that is integrative, dynamic, and generative if students 
will not invest the effort necessary to acquire information...” 

(Blumenfeld et. al. 2000, pg 375) 

The novel and varied activities found in the innovative DBS instructional 
materials may also provide opportunities to motivate student learning. However, 
exposure to interesting content and laboratory activities alone does not 
determine that measured learning will take place. The students need to find the 
tasks worthwhile and they need to put in the effort to learn the science content to 
see any changes between pre and post test scores. 

Due to the nature of the semester class schedule at the research school, 
students in version three of the DBS curriculum entered the elective course after 
having spent a semester in the 9 th grade science course. Several of the content 
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topics that were added to version 3 of the DBS curriculum were content topics 
that these students had previously covered in the first semester science course. 
In addition, the 9 th grade science teacher was also the elective teacher, and he 
choose to add several of the DBS laboratory activities to the 9 th grade science 
course. Unfortunately, this meant that the version 3 students repeated many of 
• the laboratory activities and much of the science content for batteries. These 
students did not experience novel and varied activities in their elective course. 

In examining the student scores on the attitude questionnaire, we find that 
these students in version 3 of the curriculum scored significantly (p<.001) lower 
on the post question scores of classroom enjoyment subgroup than did the 
semester 1 students. The. attitude mean of the version 3 students was 2.65 while 
the attitude means of the version one students was 3.88. These classroom 
enjoyment differences provide support for why the post-test scores on version 3 
of the DBS curriculum may not have increased as much as we expected. 
Students were not investing the necessary effort to address the demands of 
learning the added science content. Students may have seen the repeating of 
similar topics as a sign that the course was too easy and they did not engage in 
learning because they felt they had already learned the science material in their 
first semester science class. The learning that did take place may have been by 
the students who did not learn the content the first time around. Version 3 may 
have only succeeded in raising the scores for a subset of the students 
participating in the curricula enactment. 

Differences due to teacher capabilities 

The Design Based Science curriculum materials embodied many ideas 
associated with science education reform. Because of this, implementing the 
curriculum in the school requires more than just delivering the curriculum 
materials to the teachers. 

“All innovations present requirements that need to be addressed to 
achieve success. . .the materials require considerable teacher ability 
in terms of knowledge and pedagogy” (Blumenfeld et. al. 2000, pg 
155.) 

While the DBS curriculum enactments included in room teacher support and 
professional development for the processes of DBS, it did not have a 
professional development component that addressed the increase in depth of the 
science content. Students in the version 3 group showed significant gains on the 
post-test questions related to the process of DBS. However, when the in- 
classroom support was removed, and the teacher enacted the activities related to 
battery electrochemistry on his own, the researchers witnessed expressed 
misunderstandings of the concepts on the part of the teacher. These same 
content misunderstandings are seen in the errors that students made on the 
version 3 post-test items that correspond to battery electrochemistry. Clearly, 
teacher professional development must include both process and content 
support. 
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Differences due to school policies and management 

The policies in place in the school and how these policies are managed 
can affect the success of an instructional innovation in a school (Blumenfeld, et 
al. 2000). The DBS curriculum was first enacted in the fall of 1999, the first fall 
the school was operating under its new block schedule with the new elective 
courses in place. By the second semester of that year, the principle had made 
several changes to the classes to accommodate the demands of this new block 
schedule. As a consequence of these schedule changes the elective course that 
the DBS curriculum was enacted in went from having nine weeks available to 
enact the curriculum to only having five weeks available for enactment. 

In addition, the elective course was team-taught the first semester. By the 
end of the first semester the more experienced teacher in the team was pulled 
out to work in a math remediation course and the inexperienced teacher was left 
to work on his own. This meant that the increase in depth to the science content 
was accompanied by a decrease in the time that was available to address this 
content in class. Both the teacher and the students had less time to work with 
more science material. 

CONCLUSIONS AND IMPLCATIONS 

Successful gains in knowledge were accompanied with no changes in 
student attitudes when changes were made to the DBS curriculum. Differences in 
the outcomes of the posts tests for the two versions of the curriculum may be due 
to four key reasons: differences in student prior knowledge, differences in student 
motivation, differences related to teacher capabilities, and the school scheduling 
policies. These issues may have influenced how the third version of the DBS 
curriculum was enacted contributing to the lower gains in the post-test scores. 

These comparisons of the different enactments of the DBS curriculum 
suggest some implications for both student needs and reform practice when 
implementing new DBS instructional innovations. First, student and teacher 
content needs must be addressed. This can be done through a clear articulation 
of the content that will be addressed to prevent an overlap between the content 
addressed in the DBS curriculum and what is taught in the other science 
courses. Second, teacher professional development must address both DBS 
process and DBS content needs of the participating teachers. Changes to the 
content in any curriculum revision must be accompanied by changes to how the 
content capabilities of the teachers are supported. The innovation that is 
presented by DBS requires support to be successfully implemented and 
sustained. 

File: NARST2001. v March 24vFNL 
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Appendix 1: Chart summarizing Cell phone curriculum revisions 





Learning 

About 

Design 


Learning 

About 

Science 


DBS 

Cycle 


1 st 


Design goals, 
target market, 
business vision, 
case study 


Define concepts, 
Build a battery, 
battery and sound 
reading 


Partial: 
Design goal, 
Research & 
Pinups 


2nd 


Design goals, 
target market, 
business vision, 
case study 


Define, build battery 
& speaker, sound, 
battery, EM reading, 


Partial: 
Design goal, 
Research & 
Pinups 


3 rd 


Design goals, 
target market 


Define concepts, 
2 battery labs & 
readings, 1 EM 
radiation lab, 
readings, and 
problems. 


Full cycle 
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Appendix 2: Attitude Questionnaire Semantic Differential Scales: 

Directions: 



In this exercise you will rate which word best describes how you think about an 
idea. 



Example; 

Please circle the number that is nearest to the word that best describes your view of rap 
music. 

If you think rap music is important, you would circle 1 for the example below: 

Example: Rap Music is: 

Important C 1J 2 3 4 5 Unimportant 

If you don’t care about rap music, you would circle 3 for the example below: 

Example: Rap Music is: 

• Important 1 2 (3J 4 5 Unimportant 

And if you don’t like rap music, but you think it is okay, you might circle 4 for the 
example below: 

Example: Rap Music is: 

Important 1 2 3 (4J 5 Unimportant 



Please circle the number that is nearest to the word that best describes your view of science. 
Science is: 



Important 


1 


2 


3 


4 


5 


Unimportant 




Beautiful 


1 


2 


3 


4 


5 


_ ug|y„ 




Vague 


1 


2 


3 


4 


5 


Clear 




Precise 


1 


2 


3 


4 


5 


Not precise 




Frightening 


1 


2 


3 


4 


5 


Encouraging 




Simple 


1 


2 


3 


4 


5 


Complicated 




Helpful 


1 


2 


3 


4 


5 


Not Helpful 




Exhausting 


1 


2 


3 


4 


5 


Enthusiastic 




Easy 


1 


2 


3 


4 


5 


Difficult 




Useless 


1 


2 


3 


4 


5 


Essential 




Interesting 


1 


2 


3 


4 


5 


Boring 




Attractive 


1 


2 


3 


4 


5 


Unattractive 




Confusing 


1 


2 


3 


4 


5 


Ordered 




Necessary 


1 


2 


3 


4 


5 


Not Necessary 




Difficult to understand 


1 


2 


3 


4 


5 


Easy to Understand 
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Appendix 3: Attitude Questionnaire Semantic Differential Sub-groupings: 



Subgroup Construct 


Item Number 


Affective 


2,5,8,11,13 


Cognitive 


3,4,6,9,14,15 


Applicability 


1,7,10,12 
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Appendix 4: Example Attitude Questionnaire Likert Scale questions: 

Directions: Please answer the following questions about your experience in 
science class this year. Indicate your answer by circling a number 1 (not true at 
all) through 5 (very true). You can use any number on the scale. 

1 . An important reason why I do my work in science is because I like to learn 
new things. 

1 2 3 4 5 

Not true at all somewhat true very true 

2. When I show I can answer questions in science, the teacher gives me harder 
questions to think about. 

1 2 3 4 5 

Not true at all somewhat true very true 

3. When doing my work in science, I stop once in a while and go over what I 

have done. 

1 2 3 4 5 

Not true at all somewhat true very true 



4. An important reason I do my science work is because I know I will get graded 
on it. 

1 2 3 4 5 

Not true at all somewhat true very true 



7. I think learning about science is useful. 

1 2 3 4 5 

Not true at all somewhat true very true 
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Appendix 5: Subgroups for different motivation and attitude constructs 
used on Attitude Questionnaire part 2. 



Construct 


Item Number 


Personal Mastery Goal Orientation 


1,14,18,29,46 


Extrinsic Goals 


4,15,23,33,45 


Science Attitudes 


7,12,34, 44 


Technology Attitudes 


5, 25,31,40 


Thoughtfulness 


3, 6,9,17, 30, 
46, 


Thoughtfulness in Inquiry 


13,19,32,36,37 


Classroom Enjoyment 


10,22,42,47 


Science Beliefs 


28,38 


Perception of Classroom - Investigation 


8,24 


Perception of Classroom - Cooperation 


16,20 


Perception of Classroom - Involvement 


11,21 


Press for Understanding 


2,9,26,27,35, 

41 
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Appendix 6: Example Science Content Test items. 

Low cognitive Demand Science Content: 

4. Which of the following is different? 

A. microwaves 

B. visible Light 

C. Sound waves 

D. X-rays 

Medium Cognitive Demand Science Content 

6. Which of the following EM wavelengths has the highest frequency? 

A. .01 meter 

B. 1.0 meter 

C. 10 meters 

D. 100 meters 



High Cognitive Demand Science Content 

16. The following shows a discharge curve of a battery. 




17. Explain the relation between voltage and time. 

1 8. Explain the change of the slope of the curves. . 

19. Assuming that a personal computer needs at least 5.3V to operate, would 
you choose this battery for operating your personal computer? 

Yes No 

Why: 
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Low Cognitive Demand Design Process 
14. The first stage in a design project is 

A. building a prototype. 

B. locating a market need. 

C. creating a computerized model. 

D. reviewing existing technology and scientific knowledge. 
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